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ABSTRACT 

This paper compares the effects of desegregation on 
black achievement with the effects of other factors in the process of 
school learning that have recently been synthesized. The first 
section of the paper discusses techniques and guidelines for research 
synthesis, including meta-analysis. The second section presents a 
summary of. the statistical analyses of research reviews of the 1970s 
and a collection of meta-analyses of the 1980s, which reveal the 
consistently potent productivity factors in school learning and which 
further illustrate techniques for research synthesis. The third 
section assesses selection criteria for studies of school 
desegregation and achievement and compares the effects of 
desegregation— as revealed by three recent meta analyses— with the 
effects of the educational productivity factors. It is concluded that 
the amount and quality of instruction, constructive classroom morale, 
stimulation in the home environment, and other such productivity 
factors are more effective in increasing black achievement than is 
school desegregation. (CMG) 
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The purpose of the present paper is to analyze research 
on the impact of school desegregation on academic 
achievement. More specifically, the particular emphasis of 
this paper is the comparison of the effects of desegregation 
with those of other factors in the process of school 
learning that have been recently synthesized. 

The paper is divided into three sections. The 
remainder of this first section discusses techniques and 
guidelines for research synthesis including meta-analysis. 
The second section presents a summary of the statistical 
analyses of research reviews of the 1970's and a collection 
of meta-analyses of the 1980's, which reveal the 
consistently potent productivity factors in school learning 
and which further illustrate techniques and guidelines for 
research synthesis. The third section assesses selection 
criteria for studies of school desegregation and 
achievement, and compares the effects of desegregation—as 
revealed by three recent meta-analyses--with the effects of 
the educational-productivity factors. 

Research Synthesis 

The present is an extraordinary time in the history 
of education because research syntheses are demonstrating 
the consistency of educational effects and are helping to 
put teaching and other determinants of learning on a sound 
scientific basis. Research synthesis is an attempt to apply 
scientific techniques and standards explicitly to the 
evaluation and summarization of research; it not only 
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statistically summarizes effects across studies but also 
provides detailed, replicable rationales and descriptions of 
literature searches, selection of studies, metrics of study 
effects, statistical procedures, and overall results as well 
as those that call for exception with respect to context or 
subjects by objective statistical criteria CGlass, 1977; 
Cooper $ Rosenthal, 1980; Jackson, 1980; Walberg $ Haertel, 
1980; Glass, McGaw, $ Smith, 1981; and Light $ Pillemer, 
1982). Qualitative insights may be usefully combined with 
quantitative synthesis (Light $ Pillemer, 1982); and 
quantitative results from multiple reviews and syntheses of 
the same or different topics may be compiled and compared to 
estimate their relative magnitudes and consistencies 

(Walberg, 1982). 

Research synthesis is not merely statistical analysis 
of studies. Jackson C1980) discusses six tasks comprising 
an integrative review or research synthesis: specifying the 
questions or hypotheses for investigation; selecting or 
sampling the studies for synthesis; coding orjrepresenting 
the characteristics of the primary studies; analyzing, or 
meta-analyzing CGlass, 1977) or statistically synthesizing 
the study effects; interpreting the results; and reporting 
the findings. 

Although these tasks seem obviously necessary to 
encourage replication of reviews, Jackson found only 12 out 
of 87 recent reviews in prominent educational, 
psychological, and sociological journals that provided even 
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a cursory statement of methods. The basic idea behind much 
good advice in Jackson's paper is that the methods of review 
and synthesis should be explicit to enable other 
investigators to attempt to replicate the synthesis. 

Explicit methods concerning quantitative synthesis, 
however, inevitably call for statistics, and two are most 
often employed- -the vote count or box score, and the effect 
size (Glass, 1977). The vote count is easiest to calculate 
and explain to those who are unaccustomed to thinking 
statistically; it is simply the number of percentage of all 
studies that are positive, for example, in which the 
experimental exceeded control groups or the independent 
variable correlated positively with the dependent variable. 

The effect size is the difference between the means of 
the experimental and control groups divided by the control 
group standard deviation; it measures the average 
superiority Cor, inferiority, if negative) of the 
experimental relative to the control groups (for cases in 
which these statistics are unreported, Glass (1977) provides 
a number of alternate estimation formulas). If education 
had uniform ratio variables such as time and money as in 
economics or physical measures in natural sciences such as 
meters and kilograms, effect sizes would be unnecessary; it 
could be said, for example, that the experimental groups 
grew .42 comprehension units in reading history on average, 
and the control group grew .22 units without crude post hoc 
standardization for comparability required in meta-analysis. 
Effect sizes permit a rough calibration of comparisons 
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across tests, contexts, subjects, and other characteristics 
of studies. The estimates, however, are affected by the 
variances in the groups, the reliabilities of the outcomes, 
the match of curriculum with outcome measures, and a host of 
other other factors, whose influences, in some cases, can be 
estimated specifically or generally. Although effect sizes 
are subject to distortions, many of which may counterbalance 
one another, they are the only means of comparing the size 
of effects in primary research that employs various outcome 
measures on non-uniform groups. They are likely to be 
necessary until an advanced theory and science of 
educational measurement develops ratio measures that are 
directly comparable across studies and populations. 

General iz ability 

The generality of the results of the synthesis can be 
divided into questions of extrapolation and interpolation: 
Do the synthesized results generalize to other populations 
and conditions, particularly to those that have not been 
studied or for whom the results are unpublished? And, do 
the results generalize across populations and conditions for 
which results are available? Extrapolation may be invalid 
beyond published studies because journal editors favor 
positive, significant studies. Smith (1980) estimates from 
several syntheses that mean effect sizes in unpublished 
work, mainly doctoral dissertations, are occasionaly ' larger 
but average about a third smaller than those in published 
studies . 
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Rosenthal (1980), on the other hand, shows that, given 
the great statistical significance of collections published 
studies, the probability of null effects being established 
by unpublished studies is minimal. Furthermore, both the 
low reliability of educational measures and low curricular 
validity Ccorr espondance of what is taught and what is 
tested on outcome measures) diminish the estimates of 
relations between educational means and ends. Less than 
optimal reliability and validity, which leads to 
underestimates of effects, probably more than compensate for 
publication bias; but more empirical and analytic work is 
needed on these factors to determine their general and 
specific influences on synthesis results. 

Interpolation 

The interpolation problem can be readily solved by 
additional calculations. The most obvious questions in 
quantitative synthesis concern the overall percentage of 
-c irve results and their average magnitude. But the next 
questions should concern the consistency and magnitude of 
results across student and teacher characteristics, 
educational treatments "and conditions, subject matters, 
study outcomes, and validity factors in the studies. These 
questions can be answered by calculating separate results 
for classifications or cross-classifications of effects. 

The results may be compared by objective statistical 
tests (such as T, F, and regression weights in general 
linear models). They permit conclusions on such matters as 
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the overall effectiveness of treatments as well as their 
differential effectiveness on categories of students in 
various conditions on different outcomes. Notwithstanding 
the frequent claims by reviewers for differential effects on 
the basis of results of a few selected studies, most 
research syntheses yield results that are robust and roughly 
consistent across such categories. Such robustness is 
scientifically valuable because it indicates parsimonious, 
law-like findings; it is also educationally valuable because 
educators can apply robust findings more confidently and 
efficiently rather than using complicated, expensive 
procedures, tailor-made on unproven assumptions to special 
cases . 

A number of useful methodological writings are 
available. Glass (1977) provides a concise introduction to 
statistical methods; and Glass, McGaw, and Smith's (1981) 
book presents a comprehensive treatment. Jackson (1980) and 
Cooper (1982) discuss tasks and criteria for integrative 
reviews and research syntheses. Light and Pillemer (1982) 
decribe methods for combining quantitative and qualitative 
methods. Walberg and Haertel (1980) present a collection of 
eight methodological papers by Cahen, Cooper, Hedges, Light, 
Rosenthal, Smith and others and thirty-five substantive 
papers mostly on educational topics. In forthcoming work, 
Larry Hedges of the University of Chicago and Barry McGaw of 
Murdoch University (Australia) offer firmer statistical and 
psychometric footings for quantitative synthesis. Important 
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guidelines for research synthesis that may be found in these 
works are further discussed and illustrated in the remaining 
sections . 

Educational Productivity Factors 
A Review of Revie ws of T eaching Effects 

The year 1980 marked a trans it ional period when 
investigators recognized the shortcomings of the traditional 
review and the advantages of more objective, explicit 
procedures for evaluating and summarizing 'research. Yet 
reviews still have a place, and much can be learned from 
them. Waxman and Walberg C1982) examined 19 reviews of 
teaching process-student outcome research published during a 
recent decade that critically reviewed at least three 
studies and two teaching constructs; they described their 
methods, compared their conclusions, synthesized them, and 
and pointed out the implications for future reviews, 
syntheses, and prior research. 

The 19 reviews reflect the inexplicit, varied, and 
vague standards revealed by Jackson f s (1980) analysis of 87 
review articles in prominent educational, psychological, and 
sociological journals. None of the reviews, for example, 
described their search procedures , and only one stated 
explicit criteria for inclusion and exclusion of primary 
studies. Comparative analysis of the studies, moreover, 
revealed that the reviewers failed to search diligently 
enough for primary studies or to state the reasons for 
excluding large parts of the research evidence. Among the 
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five reviews that covered positive reinf orcment such as 
praise and feedback in teaching, only six studies were 
covered in the most comprehensive review in contrast to the 
39 listed in Lysakowski and Walberg's (1981) synthesis. 
Such arbitrary selection of small parts of the evidence, of 
course, leaves the reviews open to systematic bias and means 
that the reviews and their conclusions cannot be replicated 
in a strict sense because their methods are undescribed. 

Although the reviews purported to be critical, their 
coverage of the 33 standard threats to methodological 
validity (Cook § Campbell, 1979) was spotty and haphazard. 
In 95.4 percent of the possible instances, the reviews 
ignored specific threats. External validity (interaction of 
teaching treatments with selection, setting, and history) 
was relatively well covered, perhaps reflecting the search 
and claims for apt i tude- treatment interactions of the 
1970 f s; but the serious problem of internal validity such as 
reverse and exogenous causes in correlational studies were 
almost wholly ignored. Indeed, there appeared an odd 
tendency to select correlational studies rather than 
experiments for review. 

Despite these problems, however, a statistical 
tabulation of the conclusions of the reviews shows 
substantial and statistically-significant agreement that 
five broad teaching constructs — coghiti ve cues, motivational 
incentives, engagement, reinforcement, and management and 
climate--are positively associated with student learning 
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outcomes (see Table 1). These tabulations, moreover, are in 
close agreement with quantitiative syntheses of large, 
systematic collections of primary studies discussed in a 
subsequent section. 



Insert Table 1 about here 



Current Research Syntheses 

To characterize quantitative syntheses of educational 
research completed since 1979, sixteen were found in 1982 by 
scanning publications of the American Educational Research 
Association and writing to the members of "the invisible 
college 11 of about 100 scholars that meet annually to present 
and discuss research on teaching. A more systematic search 
in late 1982 using Dissertation Abstracts, Social Science 
Citation Index, Education Index, computer retrieval, and 
references in recent publications indicates that these 
syntheses plus those discussed in subsequent sections of 
this chapter represent about three- fourths of those 
completed in education thusfar in the 1980s. (An analysis 
of a more complete corpus is underway by the present author 
and colleagues, but the increasing number of syntheses malces 
exhaustive coverage an elusive goal.) 

Table 2 suggests a number of ■ ins tructi ve points for 
both educational practice and research synthesis. It 
provides, for example, an empirical answer to the 
coincidence of vote counts and effect sizes. Every mean 
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effect size that was positive also had a vote count greater 
than 50 percent; every negative effect size had a vote count 
less than 50 percent. Thus, as may be expected from normal 
distributions, consistently positive findings will yield 
positive average results (the next section shows that much 
of the variance in effects can be predicted by regression 
from counts). The likely explanation for the uniform 
association is that strong causes produce results consistent 
in sign. Indeed, the only cases in which the association 
can be reversed are skewed distributions in which a few very 
strong positive results are sufficient to pull the mean 
above zero from a cluster of small effects, more than half 
of which are negative Cor vice versa). 



Insert Table 2 about here 



The first two syntheses grouped under Teaching 
Stategies in Table 2 show fairly close agreement with 
respect to the consistency of cooperative learning. Johnson 
and others (1981) categorized their results by comparisons 
of four treatment variations (cooperative, competitive, 
group competitive, and individualistic), whereas Slavin 
' (1930) categorized his results by outcomes. Cooperative 
learning obviously produces superior results; but it would 
be useful if journal editors would allow research 
synthesists space to report average results by more standard 
classifications of independent and dependent variables and 
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study conditions to facilitate compar isions of replicated 
syntheses such as these two. 

The next two syntheses raise important, unresolved 
methodological questions, Becker and Gersten's (1982) 
synthesis indicated a small average effect of direct 
instruction in several sites, but all effect sizes came from 
the same study. Although teachers in the various sites may 
have been independent actors, methodological bias can make 
the effects non-independent from a statistical point of 
view, and independent replications by different 
investigators would be in order to a provide a more 
definitive answer. Pflaum and others (1980) found no 
average superiority of different reading methods but a 
substantial advantage in learning outcomes of experimental 
over control groups no matter what the reading method 
employed. Although Hawthorne effects could be discounted by 
the synthesis, the increased energy and attention devoted to 
tasks, by teachers in experimental groups rather than 
putative treatments themselves may partly account for 
superior results in teaching-methods and other educational 
studies. 

Table 2 includes two rough replications that indicate 
substantial agreement in results despite large variations in 
study search, selection, and-numbers. Hansford and Hattie's 
(1982) and Findley and Cooper's (1981) syntheses of 
correlations of self-concept and locus of control with 
achievement and performance differ only slightly in the 
second decimal place in both the vote counts and average 
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correlations. Carlberg and Kavale's (1980) and Ottenbacher 
and Cooper's (1981) syntheses agree that the effects of 
mainstreaming (federally- encouraged efforts in the United 
States to mix regular and cognitively, emotionally, and 
physically handicapped children in the same classes) are 
inconsistent and probably near zero. 

Two syntheses show curvilinear effects of independent 
variables on educational outcomes. Smith and Glass (1980) 
found that the benefits of reduced class size are larger at 

\ 
i 

the smaller ranges of one to 10 members than they are at 
higher ranges; for example, the measureable cognitive and 
affective outcome differences between classes of 20 and 60 
appear trivial. Similarly, Williams and others (1982) found 
decreasing achievement with departures from 10 weekly hours 
of leisure-time television viewing such that estimated 
differences in achievement between children who watch about 
30 hours--an average number--and 60--a large amount--are 
miniscule. 

Other effects are summarized in the table, and the 
reader is referred to the original syntheses for details 
that are not' discussed here. Overall the results indicate a 
large range of effects, which, if replicated in further 
primary research and syntheses, would have fairly definite 
implications for choosing policies and practices that seem 
likely to have consequential effects on raising educational 
outcomes. 

The Michigan Progra m , 
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Chen-Lin and James Kulik lead a vigorous group of 
research synthesists at the University of Michigan, which 
included Peter Cohen, now of Dartmouth. The group has been 
unusually productive of hi gh-q~uali ty syntheses first in 
higher education and later in secondary- school research. 
Personal communications with the group reveal that their 
team approach, much like that described by Shulman and Tamir- 
(197 3) in the Second Handbook of Research on Teaching, 
accounts in part for the quantity and quality of work. 

t 

James Kulik kindly prepared Table 3 according to the 
present author's specifications. It shows the results of 
eleven syntheses completed by the Michigan group by the end 
of 1981. Like the sixteen syntheses by other investigators 
discussed in the last section, those in Table 3 show a 
number of consistent moderate to large effects that can help 
to put high school and college teaching on a firm scientific 
basis. 



Insert Table 3 about here 



Kulik's results also permit an estimate of the mean 
size of effects from vote counts. The regression equation, 
ES = -.403 ♦ .008 (% Positive), accounts for 76 percent of 
the variance in the effect sizes. The corresponding 
equation for the syntheses in Table 2 for which both indexes 
are available, ES « -.761 ♦ .015 (%), accounts for 59 
percent of the effect-size variance (the correlational 



ERIC 



14 is 



Final Report 



results assume both causality and a one-unit increase in the 
independent variable). Both equations forecast near zero 
effect sizes for vote counts of 50 percent; but the higher 
slope for the results in Table 2 forecast larger effects 
than do the Michigan data; at vote counts of 75 percent, for 
example, the respective forecasts are .36 and .20. Thus the 
size of the regression slope is unstable across samples, and 
more intensive analyses of the complete corpus of syntheses 
are in order. 

The two data sets also permit separate empirical 
estimates of the distributions of vote counts and effects. 
The mean Cand standard deviations) of Michigan and other 
estimates of the vote counts are respectively 67 and 64 Cand 
19 and 16); the mean effects are respectively .17 and .22 
Cand .19 and .31). Assuming normal distributions of 
effects, empirical norms for vote counts and effect sizes 
can be set forth on the basis of the averages of these 
statistics; for example, the middle two-thirds of the 
effects in the recent educational research sampled range 
from about -.05 to .45. It could be said that effect sizes 
of .20 are average, and those above .45 are large and exceed 
about 84 percent of those typically found in educational 
research. Similarly, vote counts of 67 and 85 percent might 
be provisionally taken as average and large. These norms 
are, of course, very rough and preliminary, but they are 
based empirical results rather than opinion and may be 
useful in gauging present and future results until larger 
normative samples are analyzed. 

ERIC 15 16 



Final Report 



Syntheses of Bivariate Productivity Studies 

A group at the University of Illinois at Chicago has 
concentrated on synthesizing research on nine theoretical 
constructs that appear to have consistent causal influences 
on academic learning: student age or developmental level, 
ability (including prior achievement), and motivation; 
amount and quality of instruction; the psychological 
environments of the class, home, and peer group outside 
school; and exposure to the mass media (Walberg, 1981). The 
group first collected available vote counts and effect sizes 
in the review literature of the 1970 f s and then conducted 
more systematic syntheses directly on the nine factors. 
This section summarizes both efforts. 

Synthesis of reviews of the 1970 f s . Walberg, Schiller, 
and Haertel (1979) collected reviews published from 1969 to 
1979 on the effects of instruction and related factors on 
cognitive, affective, and behavioral learning in research 
conducted in elementary, secondary, and college classes and 
indexed in standard sources. The vote counts for the corpus 
of reviews are shown in Table 4. 



Insert Table 4 about here 



The vote counts should be cautiously interpreted 
because not only may journal editors more often select 
studies with positive results but also reviewers may select 
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positive published studies for summarization. Neither 
editors nor reviewers ordinarily state explicit policies on 
these important points. Subsequent, more systematic 
syntheses, nonetheless, have generally supported 
traditional reviews; and it would be wasteful to ignore the 
labors of the last decade of effort, even though it may only 
be considered a starting point for subsequent work. 

Notwithstanding the possible double bias in the vote 
counts Csee earlier sections on counter-biases), the results 
in Table 4 are impressive. A majority of the variables in 
the table were positively associated with learning; in 48 or 
68 percent of the 71 tabulations, 80 percent or more of the 
comparisons or correlations are positive. Although all of 
the variables are candidates for synthesis using systematic 
search, selection, evaluation, and summarization procedures, 
it appears that the 1970's produced reasonably consistent 
findings that are likely to be confirmed by more 
comprehensive and explicit methods of the present decade. 

Syntheses of Product ivrty_ Factors. The Chicago group 
also carried out syntheses of the nine factors using methods 
discussed in previous sections of this chapter. The 
National Institute of Education supported the syntheses of 
- learning research in ordinary classes, grades kindergarten 
through twelve. A separate grant from the National Science 
Foundation on science learning, grades 6 through 12, 
permitted more exhaustive, intensive search for unpublished 
work and an advisory group of science educators and research 
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methodologists as well as a semi- independent replication of 
the results for several of the factors. A summary of the 
findings is shown in Table 5. 

Insert Table 5 about here 



All of the effect sizes (including mean contrasts and 
correlations) are in the expected direction. The mean 
effects for the two samples of studies are similar in 
magnitude, which suggests generality or robustness of 
effects across more and less intensive methods of synthesis. 
In particular, the syntheses of quality of instruction 
including cues, participation, and reinforcement of about 
1.0 and .8 in general grades K-12 and in science grades 6-12 
support the conclusions . of the 19 reviews discussed in a 
previous section (see also Table 1). Despite these 
corroborations of findings, of course, independent 
replications of the syntheses as well as new- and probing 
experimental studies are needed. 

Syntheses of M ultivariate Studies 

The Chicago group also conducted multivariate analyses 
of the productivity factors in samples of from two to three 
thousand 13- and 17-year-old students who participated in 
the mathematics, social studies, and science parts of the 
National Assessment of Educational Progress (see, for 
example, Walberg, Pascarella, Haertel, Junker, and 
Boulanger, 1981, 1982). These survey analyses complement 

is is 
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small-scale correlational and experimental studies in 
providing on representive national samples data on fairly 
comprehensive sets of the productivity factors, each of 
which may be statistically controlled for the others in 
multiple regressions of achievement and subject-matter 
interest. 

Such analyses allow a simultaneous assessment of 
qualities and amounts of instruction and the other factors 
in the production of learning. Since the factor levels are 

t 

reported as experienced by individual students, the analysis 
are sensitive to m i cr o - var i at i ons in the multiple 
environments of the school, peer-group, home, and mass media 
to which each student is exposed. 

Although the sets of variables available in the 
National Assessment can be used to assess possible exogenous 
causes because they are measured and can be statistically 
controlled in regression equations, the measures are cross- 
sectional for individuals. Therefore, they cannot 
effectively rule out reverse causation such as learning as a 
cause of motivation and more stimulating teaching. Another 
shortcoming of the data is that parental socioeconomic 
status serves as a proly for ability and prior achievement. 

As pointed out above, nonetheless, the strengths of the 
National Assessment data complement those of small-scale 
bivariate studies that typically control for only, one or two 
of the factors. If syntheses of both data sources point in 
the same direction, then more confidence can be placed in 
the conclusions. 
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Table 6 shows that the factors, when controlled for one 
another, are suprisingly consistent in sign, significance, 
and magnituide across subject matters, ages, operational 
measures of the factors, and independent national samples. 
The median standardized regression weights and squared 
multiple correlations, shown in the last row, reveal the 
small to moderate effects of the factors when controlled for 
one another and sizable amounts of variance accounted for 
even without ability and pri'or achievement measures. 



Insert Table 6 about here 



Syntheses of Open Education Research 

Open education is an elusive concept, now dismissed by 
many educators, but one that research synthesis now 
illuminates. The history of efforts to synthesize its 
effects is instructive about: the dangers of basing 
conclusions, policies, and practices on single studies; 
replication and improved methods of syntheses, and a 
shortcoming of much of the research discussed above that 
employs grades and standardized achievement as the sole 
outcomes of teaching. 

From the start, open educators tried to encourage 
educational outcomes that reflect school-board goals such as 
cooperation, critical thinking, self reliance, constructive 
learning attitudes, life-long learning, ani other goals that 
evaluators seldom measure. Raven's (1981) summary of 
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surveys in Western countries including England and the 
United States shows that educators, parents, and students 
rank these goals far above standardized test achievement and 
grades . 

A synthesis of the relation of conventionally-measured 
educational outcomes and adult success, moreover, shows 
their slight association CSamson and others, 1982). Thirty- 
three post-1949 studies of physicians, engineers, civil 
servants, teachers, students^ in general, and other groups 
show a mean correlation of .155 of these educational 
outcomes with success indicators such as income, self-rated 
happiness, work performance and output indexes, and self-, 
peer-, and supervisor-ratings of occupational effectiveness. 
These results should challenge educators and researchers to 
seek a balance between continuing motivation and skills to 
learn and perform well on new tasks as an individual or 
group member on one hand and mastery of teacher- chosen, 
textbook knowledge that may soon be obsolete or forgotten on 
the other. 

Perhaps since Socrates, however, arguments over 
student-centered and teacher- centered education have 
remained so polarized, polemical, and pervasive that 
educators find it difficult to stand firmly on the high 
middle ground of balanced, joint, or cooperative 
determination of the goals, means, and evaluation of 
learning. Progressive education, the Dalton and Winnetka 
plans, team teaching, the ungraded school, and other 
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innovations in this century held forth this ideal but 
gravitated toward authoritarian teaching or permisiveness 
and could not be sustained. Although open education, too, 
faded from view, it was more carefully researched; and 
syntheses of it may help prepare educators for evaluating 
future efforts. 

Three Syntheses of Op_en Education. Horwitz (1979) 
first synthesized about 200 comparative studies of open and 
traditional education by tabulating vote counts by outcome 
category. Although many studies yielded non-significant or 
mixed results especially with respect to academic 
achievement, self concept, anxiety, adjustment, and locus of 
control, more positive results were found in open education 
on attitudes toward school, creativity, independence, 
curiosity, and cooperation. 

Peterson (1979) calculated effect si^es for the 45 
published studies. She found about -.1 or slightly inferior 
effects of open education on reading and mathematics 
achievement; .1 to .2 effects on creativity, attitudes 
toward school, and curiosity; and .3 to .5 effects on 
independence and attitudes toward the teacher. 

Hedges, Giaconia, and Gage (1981) synthesized 153 
studies including 90 dissertations using an adjustment of 
Glass's effect-size estimator which is slightly biased 
especially in small samples. The average effect was near 
zero for achievement, locus of control, self concept, and 
anxiety; about .2 for adjustment, attitude towards school 
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and teacher, e.Tiosity, and general mental ability; and 
about .3 for cooperativeness , creativity, and independence. 

Despite the "differences in study selection and 
synthesis methods, the three studies converge roughly on the 
same plausible conclusion: students in open classes do 
slightly or no worse in standardized achievement and 
slightly to substantially better on several outcomes that 
educators, parents, and students hold to be of great value. 
Unfortunately, the negative conclusion of Bennett's (1976) 
single study- -prefaced by a prominent psychologist, 
published by Harvard University Press, publicized by the New 
York Times and media and experts that take that newspaper as 
their source — pa obably sounded the death knell of open 
education, even though the conclusion of the study was later 
retracted (Aitkin.. Bennett, $ Hesketh, 1981) because of 
obvious statistical flaws in the original analysis (Aitkin, 
Anderson, § Hinde, 1981). 

Components of Open Education. Giaconia and Hedges 
(1982) took another recent and constructive step in the 
synthesis of open education research. From the prior 
effect-size synthesis, they identified the studies with the 
largest positive and negative effects on several outcomes to 
differentiate more and less effective program features. They 
found that programs that are more effective in producing the 
non-achievement outcomes — attitude, creativity, and self 
concept-- sacrificed academic achievement on standardized 
measures. 

These programs were characterized by emphasis on the 
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role of the child in learning, use of diagnostic rather than 
norm-referenced evaluation, individualized instruction, and 
manipulative materials but not three other components 
sometimes thought essential to open programs- -multi - age 
grouping, open space, and team teaching. Giaconia and 
Hedges speculate that children in the most extreme open 
programs may do somewhat less well on conventional 
achievement tests because they have little experience with 
them. At any rate, it appears from the two most 
comprehensive syntheses of effects that open classes on 
average enhance several non-standard outcomes without 
detracting from academic achievement unless they are 
radically extreme. 

Synthesis of Instructional Theories 

To specify the . producti vi ty factors in further 
theoretical and operational and detail provide a more 
explicit framework for future primary research and 
synthesis, Haertel, Walberg, and Weinstein -(1983) compared 
eight contemporary psychological models of educational 
performance. Each of the first four factors in Table 7-- 
student ability and motivation, and quality and quantity of 
instruction—may be essential or necessary but insufficient 
by itself for classroom learning Cage and developmental 
level are omitted because they are unspecified in the 
models). 



Insert Table 7 about here 
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The other fopr factors in Table 7 are less clear: 
although they consistently predict outcomes, they may 
support or substitute for classroom learning. At any rate, 
it would seem useful to include all factors in future 
primary research to rule out exogenous causes and increase 
statistical precision of estimates of the effects of the 
essential and. the other factors. 

Table 7 shows that, among the constructs, ability and 
quantity of instruction are widely and relatively richly 
specified among the models. Explicit theoretical treatments 
of motivation and quantity of instruction, however, are 
largely confined to the Carroll tradition represented in the 
first four models; and the remaining factors are largely 
neglected. 

The table poses empirically-researchable theoretical 
questions; the tension between theoretical parsimony and 
operational detail, for example, suggests several: Can the 
first four constructs mediate the causal influences of the 
last four? Would assessments of Glaser's five student-entry 
behaviours allow more efficient instructional prescriptions, 
than would, say, Carroll's, Bloom's, or Bennett's more 
general and more parsimonious ability subconstructs? Would 
less numerous subconstructs than Gagne's eight instructional 
qualities and Harni s chf eger and Wiley's seven time 
categories suffice? 

The theoretical formulation of educational performance 
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models of the past two decades since the Carroll and Bruner 
papers has made rapid strides. The models are explicit 
enough to be tested in ordinary classroom settings by 
experimental methods and production functions. Future 
empirical research and syntheses that are more comprehensive 
and better connected operationally to these multiple 
theoretical formulations should help reach a greater degree 
of theoretical and empirical consensus as well as more 
effective educational practice. 

■ 

Desegregation and Educational Productivity 

As the previous section has shown, sufficient empirical 
and theoretical syntheses have accumulated during the past 
five years to point more definitively than ever before to 
the proximal, alterable factors that affect educational 
achievement. Nearly all the research has been carried out 
in natural settings such as homes and schools, and most of 
it shows generalizability across student characteristics, 
subjects, and research methods, including randomized 
assignment to experimental treatments. 

The large average magnitude and consistency of many of 
these productive factors justly provides a substantial 
amount of confidence about how educational achievement may 
be raised. Skice many of the factors and techniques have 
already been extensively employed in ordinary schools and 
found successful, inexpensive, and non-con. v -sial, it 
appears that educational achievement might be increased 
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substantially by implementing a selection of the most 
productive of the factors, say, those with effect sizes 
above .3, more extensively and intensively. The purpose of 
this section is to compare the consistency and magnitude of 
such factors to the effects of school desegregation, as 
revealed by three recent meta-analyses--Krol (1978), Cram 
and Mallard (1982), and my statistical summary of the studies 
meeting the selection criteria of the National Institute of 
Education (NIE) panel of scholars. 

Selection Criteria 

Aside from the inclusion of data only on black students 
in all three meta-analyses, Krol (1978, p. 16), Crain and 
Mahard (1982, p. 6) and the NIE panel (Schneider, Note 1) 
varied considerably in explicit criteria for study 
selection. Krol, for example, excluded, studies that lacked 
achievement measures before and after desegregation and 
those that lacked sufficient statistics to calculate effect 
sizes (pp. 83-84). Excluding studies without pretests turns 
out to be a reasonable decision because. Wortman's (Note 2) 
research shows desegregated groups are on average advantaged 
on achievement before desegregation. Thus apparent posttest 
advantages of desegregation are in part attributable to pre- 
existing differences, and pretest adjustment is required for 
valid estimation of desegregation effects. 

Crain and Mahard (1982) excluded "excluded a large 
number of papers, many of which compared students in 
racially segregated and racially mixed schools, but gave no 
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indication that a formal desegregation plan had been 
adopted" (p. 6): Because they included studies that 
employed ability (in contrast to educational achievement) as 
a dependent variable and conducted a more recent and 
exhaustive search, they used 93 studies for analysis in 
contrast to Krol's 55 (see Tables 8 and 9). 



Insert Tables 8 and 9 about here 



The NIE panal employed a number of stringent criteria 
for study rejection including the following: non-empirical 
and summary reports; studies done outside the U. S. and 
geographically non-specific; those that combined or compared 
ethnic groups, lacked contemporaneous- control or pre- 
desegregation data, or analyzed heterogenously desegregated 
groups; those with more than 35 percent attrition, majority- 
black desegregated conditions, varied exposure to 
desegregation, and non-comparable groups; those with unknown 
sampling procedures, cross-sectional data, or non-comparable 
samples at each observation point; those with unreliable or 
unstandardized instruments-, unknown test content or 
instruments, unknown test administration dates, ability 
tests as dependent variables, and non- equivalent pre- tests 
and post-tests; and insufficient statistics (Schneider, Note 

1) . Application of these exclusion criteria (Wortman, Note 

2) resulted in 19 "acceptable studies." 

Thus, all three 4ata sets are similar in including only 
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studies of black achievment. The differ chiefly in that 
Krol and the NIE panel, unlike Crain and Mahard (1982), 
exclude ability, tests, and the NIE employed stringent 
methodological criteria that resulted in a selection of 
studies only 19 percent as large as Crain and Mahard's set 
(see Table 8). 

The NIE panel may be right in specifying stringent 
selection criteria from one viewpoint: the conclusions of 
review articles are usually based upon methodologically 
acceptable studies. But, as Glass, McGaw, and Smith (1982, 
p. 226) point out, excluding studies by implicit or explicit 
selection criteria can convert empirical questions of 
research methodology to a priori assumptions. Excluding 
studies without pretests, for example, may exclude 
randomized experiments—possibly the best design in certain 
respects for probing causality and avoiding untenable 
covariance assumptions. 

If it were to be found that randomized post-test only 
designs yielded the same results as pre- test-post- test 
quasi-experiments, then greater confidence could be placed 
in the results than the results of either design by 
themseves, since the two designs are subject to different 
threats to methodological validity (Cook § Campbell, 1979). 
Because, for example, the findings on instructional 
research are generally robust and consistent across study 
features such as research methods and student 
characteristics, substantial confidence can be placed in 
their results. 

0 , ..29 
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Moreover, excluding studies on policy or substantive 
criteria may be useful to lighten the effort or to narrow 
research questions; but exclusion also restricts the 
inferences and comparisons that can be made and the policies 
that may be implied. In the Krol and NIE selections, for 
example, it will not be possible to determine whether 
desegregation has a different impact on achievement than it 
does on ability or other educational outcomes such as 
creativity, critical thinking, interest in further learning, 
and social percept iveness. In none of the three sets of 
studies, moreover, will it be possible to compare the 
effects of desegregation on Asian, black, Hispanic, and 
white students. At least for some parents, educators, 
policy makers, researchers, and others, it would be useful 
to have reliable information on these and other points. 

None of this is to argue that all studies should be 
summarized in one overall vote count or mean effect size. 
Although that statistic and its significance are of 
interest, characteristics of the studies ."such as Cook and 
Campbell's (1979) 33 threats to methodological validity, 
student characteristics such as ethnicity and grade level, 
and conditions of • desegregations such as voluntary and 
mandatory plans should be categorized, coded, and tested for 
statistical significance with studies as the units to afford 
independence as assumed in statistical inference. (If 
desegregation is working generally well according to a 
study, then students in different grades within the study 
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are likely do well, and their performance is correlated and 
not statistically independent; similarly, if students are 
doing poorly in- another study, different grades lack 
independence; therefore the means for studies, not for grade 
levels or other units, must be taken as the units for meta- 
analysis or each comparison in a study must be weighted 
inversely "to the number of comparisons in the study. 
Another reason for using study means or weighting is to 
insure that each study is given an equal weighting of one, 
not a weighting based on the arbitrary number of comparisons 
the investigator happened to make.) 

Synthesis of Three Meta-analyses 

Tables 8 and 9 show what can be validly extracted as 
the chief findings from the three meta-analysis. Table 8 
shows that three estimates of percent-positive studies vary 
between 61 and 64 percent. These percentages are in 
surpirsingly close agreement considering the widely 
different selection criteria and numbers of studies in the 

three syntheses. 

Table 9 shows that the statistical significance cannot 
be determined in two cases because the percentage of 
positive comparisons rather than studies are reported; and, 
in the NIE case, the sign test based on the number of 
studies is insignificant. By the norms of recent syntheses 
of productivity factors discussed in previous sections, the 
percentage magnitudes are neither large (85 percent) nor 
average (67 percent). The statistical significance of the 
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percentages cannot be determined in the two previous 
syntheses previously reported and is insignificant in the 
case of the NIE selection. 

The statistical significance of the effect sizes are 
mixed: indeterminate for Krol, because of comparison 
weighting; significant for Crain and Mahard; and >?ot 
significant for the set of studies acceptsble to the NIE 
panel. In none of the three cases was the magnitude of the 
effect large (.45) or average (.20). (Crain and Mahard's 
significant finding of higher effects in kindergarten and 
first grade are unsupported by Krol and reversed in analyses 
by Wortman (Note 2); and their r andomi zed- longi tudinal 
effect is insignificant with study as the unit. Thus, their 
overall average study- weighted effect size is reported in 
Table 8.) 

The results from the three meta-analyses suggest that 
the vote counts fail with some uncertainty to reach 
conventional levels of statistical significance. By 
normative standards of recent syntheses of other educational 
factors, they clearly fail with respect to percentage 
results. The effect sizes as a set are indeterminate with 
respect to significance and certainly, fail to reach 
criterion levels with respect to normative magnitude. 

Conclusion 

New techniques of research syntheses show a number tof 
potent factors for improving educational achievement that 
have proven to be consistently effective in a wide variety 
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of experimental and educational conditions. These include 
the amount and quality of instruction, constructive 
classroom morale, and stimulation in the home environment. 
It is in our national economic, social, and political 
interest to implement these factors more deeply and widely 
for all children (Walberg, 1983). In this effort, school 
desegregation does not appear to prove promising in the size 
or consistency of its effects on learning of black students. 
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Wortman, P. Personal communications. August 28, 1982; 

November 10, 12, 1982. 



34 35 



Table 1 

f.Wrnrofu vf 19 Rrvifm ftnrf 
2 QtinnhtntH* Sin9hr%t\ <j 
Ht\mnh on Turning 



Number of Gwertnfl ttmurwi 

Namrtffr of fie*tewi Omrtutlina Relation 
in Lramtnn h rtoiifte 

Mean tftotf »iw twin Quanthatit* Simh^h 

fnttmWIhf of F.tWeme Aauminft Zero 
Population tJfrtt 



SliimiUlhfH 



litteinivtt 



17 
.01 



5 
.10 



.01 



10 

HI 

.01 

.an 



15 

.10 



01 



Management 
ami Climate 



15 

15.5 
01 

1.17 



36 



ERIC 



Table 2 



[elected Post-1979. Quantitative Syntheses 



Author 
Teaching Strategies 

Johnson, naruyaraa, 
Johnson* Nel9on, and 
Stan (1981) " 



Number of Independent and 
Studies ,; Bopendent Variables 



Mean 

Correlation Percent 

or Effect Positive Comments 



Slavin (1980) 



Becker a Gersten 
(1982)) 



Him, Halberg, 
Karagianee, and 



122 Effects of cooperation, in- 
tergroup and interpersonal 
competition, and individual 
goal efforts en achievement 
and productivity 



28 Effects of educational programs 
for cooperative learning 



Effects of Direct Instruction .23 
Follot? Through on later 
achievement (7 sites on 2 
occasions, fifth and sixth 



.60 



00 


54 


78 


76 


37 


68 


76 


83 


59 


81 


,03 


47 




81 




78 

* 95 




65 



Effects of different methods 
teaching reading on learning 



76 



Cooperative vs. group competitive 
Cooperative vs. competitive 
Group competitive va, ewpmtlfft 
Cooperative vs. individualistic 
Group c^titive va. inflifiduallsM 
Competitive vs. individualistic . 

Currieului o speeif ic tests 

Standardised tests 
Race relations 
Mutual concern 

Effects lar^ tot mathematics 
problem solving and for fifth 



Although flauthornfl effects could tc 
discounted, experimental pups 
generally did substantially tetter 
than contfolot aound-aymtol blendic 
ms one standard deviation higner ; 
than other treatments. 



Table 2 (page 2 of 3) 



Author 

Teaching Skills 

Luiten# Ames, and 
Anderson (1980) 



flumberof Independent and 
Studies Im pendent Variables 



135 Effects of advance organ- 
izers on learning and 
retention 



itedfield and tous flMU 20 
(1981) 



Vinson (1980) 



Mean 

Correlation Percent 

or Effect Positive Comments 



Effects of higher and lowr 
cognitive questions 



14 Effects of praipe on 
achievement 



.23 



.73 



63 



Effects larger on 20* days retentlei 
higher achievers, college students, 
and '/hen presented aurallf 

Higher questioning effect! greater 
training than in skills study and 
in more valid studios 

Praise allghtly roere effective for 
lowr socioeconomic groupBj primary 
grades,, and in mathematics 



Other Studies 
Butcher (1981) 



Findley 
(1981) 



4? 



Colosimo (1981) 24 



Effects of mieroteaching «84 

lessons on teaching perfor- .56 

mance of secondary and »W 

elementary education students .35 



Effects of practice and be- 
ginning teaching on self 
attitudes 



Correlations of locus of 
control and achievement 



•.29 



79 



Secondary specific skills 
Secondary questioning skills 
Elementary specific skills 

Elementary questioning skills 

initial esperleree aaswlated «tth 
greater authoritarianism and »lf 
doubtiinner-city experience m 
negative 

Correlations hlglwr among inalesr 
for adolescents In eonstrast to 
children and adult groupsi for spa 
ciflc control measuresi and for ot 
jective achievement 



ERIC 



33 



40 



Table 2 (page 3 of 3) 



author 



Number of Independent and 
Studies, g pendent Variables 



Mean 

Correlation Percent 
or Effect Positive 



fihri and (tattle 128 Correlation of self-concept .21 
fl " a and achievement/performance 



Carldter? and tovale 30 Effects of special versus - 
(1980) regular classes 



.12 



Ottorbacher and 



43 



Effects of class placement «05 
of mentally retarded students -.07 
on social adjustments 



Smith and 
, (1980) 



59 



Effects of class size on 
attitudes, climate, and 
instruction 



.49 



miliums, Haertel, 
Haertel, and walberg 



ft correlations of leisure 
time television and 



-.05 



illllflon and Putnam 
(1982) 



32 Effects of pretests on 
outcomes 



.17 



61 
46 



34 



57 



Comments 

Correlations higher for hign school 
students in contrast to elementary 
and collegei higher ability students) 
specific rather than global self • 
concept! and verbal achievement 



Effects positive for learning dis- 
abled and behavior disordered and 
negative for slow learners and men- 
tally retarded 

Special -class vs, regular elaei 
' * class vs. resource class 



In contrast to small mean 
01 for achievement, moderate effect 
observed, which were larger on tea^ 
than students, younger students, una 
for studies before 1969 

Effects negative at ratio d lose,: 
than 5 or greater than 15 tours f*r 
*ek and stronger for girls and 
higher ability groups 



effects greater for 
personality outcomes, for treaw 
Ulna between 2 and 30 days, and 



lasting between 
for randomized 



i Tabid 

»» ...... n. mmm mm « « * Weh " m '' 

Center for Beaearch en learning ami Teaching 




deport 



Variable 



variable 




spE, 

aecendary teaching |tm ^ iflMp(J 
subject wetter 



MuTBtrtjr Positive mean $0 
19 



U 



Miemster rating Change on tint n 



feedback to 
teaehor va, 



ratings. 



Class rating of Glees achievement 
instructor quality on f nai 



0.10 0.38 
64 ,0.14 0.H 

91 0.39 0.41 



Comment! 



0.43 0.13 



Iff lets.*** greater entn 
teachers received consulting 
help along »lth rating 
feedback . 

Correlation wo higher 
,m teachers wre feeul v 
(not teaching saslstants). 
rton HI teste «ere 
graded by I cciwon 
grader, end ihen 
etudente rated teechirl 
after receiving 9 fB(,fi '' 



Cohen. foiling, 

m m o^lone.^ 



Achievement on 69 


9T 


0.19 


0.41 


final examination 








Student rating of 19 


39 


•0.08 


0.66 


course quality 








Courae completion 10 


30 


•0.09 


0.23 



•cnivtwnini ' . 

stronger to more recent 
atudiee. in studies 
universities, • ehen 
different teflchire teugnt 
viaual*bBoed 9 control 



table (Continued) 



Report 



Independent 
variable 



Dependent 
variable 



Studies tffeet Sin 
Number Positive Wean SO 



Cements 



•J. Rum, Cohen, 
ft tbeltn? (1980) 



J, But Ik, 
C. KU1U, 
ft Cohen (1979a) 



Ppogrsmmed vo. 
conventional 
college teaching 



PersonsWsd System 
of instruct Ion vs. 
conventions! 
eel lege teaehing 



achievement on 56 71 0.24 0.92 
final examination 

Course completion 9 61 -0.06 0,2? 

.enlevement en 61 94 [9M) 0.39 
flnaUxsmlnatlon v - 

Course completion 2? 3? -0.10 0.30 

Rating of course H 91 0.46 0.69 
quality 



achievement effects 
Mre stroller In .., 
more recent studies. 



achievement effects 
differed Oy subject end 
eero stronger ehsn 
different tesehers teugnt 
PS1 and control elssses, 
end Wien control 
etasseo eontolned PSI 
features. 



•J. Ml lk, 
C. «u!1k, 
6 



J, Rut Ik, 
C, Rut Ik, 
ft Cohen (1980) 



ludiO'tutoriel 
vs. conventional 
college teaching 



Computer-tossed 
vs. conventional 
college teaching 



achievement on 42 
final examination 

Course completion 22 



of course 6 



quality 



achievement en 94 
final examination 

Course completion 13 

fin ting of course 11 
quellty 



69 0.20 0.43 

§2 -0.10 0.3? 

SO 0.12 0.92 

69 0.29 0.61 

46 0.01 0.30 

n 0.24 0.92 



achievement effects 
vers ctrongsr In 
studies found In 



achievement effects 
vere stronger vhen 
different teecners 
taught computerised 
and control el esses, 



ffcrtem 

954 

97.* 

as? 

660 
64.2* 
72.? 
60.0 

9B.1 

95.2 
96.7 

100.0 
1000 
100.0 

1000 
1000 
100.0 
05.7 
fi7.5 
63.3 

looo 

7U6 



Table 4 (Continued) 



ftaeirch Top** 



No. «f 



Percent 

flftitivt 



Kychotofkil inccniim and engagement 
Tetchcr iruciio aiidem^ 
Teacher remforcement of nuflent 
Teacher engagement of eUu in teuon 
Indicia) ttudcnt engagement m tewon 
Optn n. traditional education on: 
Achievement \ 
Creativity \ 
fielf-conccpt 
Attitude toward achoot 
Curcnity 

fe»|f-dcicnninauon N 
Independence 
freedom from antfetj 
Cooperation 
rrogrammed tnitmciton cm teaming 
• Adjunct queitioni on teaming* 
After tew cm recall 
After teat cm tramfct 
before teat on recall 
Before teat on trantfer 
Advance organitcr* cm teaming 
AnaWtfc trvtton of inatmcuon on achievement 
Direct inwructicm cm achievement 
Lecture v*. disunion on: 
Achievement 
Retention 

&Jdem.t" «n»truaor<entcred disunion on: 
Achievement 
Undemanding 

»2££ conceptual ^ « 
taial-pueholosical climate and teaming 
Coheuveneu 
Satiifaction 
Difficulty 
formality 
Coal direction 
Democracy 
Environment 

Speed 
Diveniu 
Competition 
friction 
Cliqucneti 
Apathy 

Ddorftamaation 
favoritism 
Motivation and teaming 
Social clan and teaming 
Home environment on: 
* Verbal achievement 
Math achievement 
Intelligence 
fteading fain* 
Abiliit 



10 
16 
6 
15 



16 
• 

t 



22 
4 



IIHMI 

ft7.ft 
100.0 
Uio.o 



26 




12 


limn 


17 


Rlt.2 


2n 


921 


6 


ItnM 


7 


R5* 


iy 


94* 


a 


S7.: 


6 


|U0.< 


57 


no.7 


3* 


97.* 


55 


745 


13 


76< 


17 




32 


67. 


4 


100 


4 


too 



17 
17 
16 
17 
15 
14 
15 
14 
14 
9 
17 
13 
13 
17 
13 
232 
620 

SO 
22 

to 

6 
6 



101 
At 



Table 5 

Cemktwns and Effect Sim for Ntne+Foetort 
m Relation to School Learning 



Factor 



Number 
of 



flnulu af>d Comment 



1 nit r union 
Amount 



Quality 



Socul-piKhetogkal Environment 
Educational 1$ 



Home 



Media-TV 



Peer group 



Aptitude 

Agc^Hrlopmcnt 



Ability 



Motivation 



51 CorreUtioni fange from .15 to .71 whh a median of .40; parual 
correUtioni controlling for ability tocteeconomic tutui. and 
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Table 8 

Effects of Desegregation on Black Achievement 
in Three Syntheses 



Source 



Positive 
Resvfl .i> 
Percent 



Zfect Sizes 

Standard 
s^ean Deviation 



Comnents 



Krol (1978) 61 -16 



41 Based on 71 comparisons in 55 
studies, grade level, mathema- 
tics and verbal achievement, and 
program-duration differences 
tested and found insignificant. 



Crain § 

Mahard 

(1982) 



62 



.10 



25 Percent calculated as sum of 173 
positive and half of 50 non- sig- 
nificant comparisons oi 
comparisons in 93 studies; 
effect- site mean based on /u 
studies. With studies as 
units, significantly larger 
effects in kindergarten 
and grade one were found. 



"Acceptable 
Studies" 



64 



.13 



24 Since the pretest advantage of 
desegregated groups over con- 
trol groups was .18, results 
are calculated for 11 study- 
weighted moans of posttests ad- 
justed for pretests. 
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Table 9 



Inferences from Three Syntheses 
About the Effects of Desegregation on Black Achievement 



Krol (1978) 



Percent-Positive Studies Average Effect Sizes 



Significance Magnitude 
(.05) (671) 

f No 



Significance Magnitude 
(.05) (.20) 

? No 



Crain § 
Mahard (1982) 



No 



Yes 



No 



"Acceptable 
Studies" 



No 



No 



No 



No 



Conclusion 



No? 



No 



No 



Note-The criteria for inferences are as follows: The significance 
required is the standard .05 level calculated for a sign test for a SO- 
SO split for positive vote counts, and a T test for the difference of 
the mean effect size from zero, when possible, on independent units of 
analysis, that is. studies not comparisons. The magnitude criteria are 
67 percent of the studies positive and an average effect size of .20. 
for which the desegregated students would exceed 58 percent of the 
control-group students. 
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